Rapid Evaluation of Multiple Density Models

When highly-accurate and/or assumption-free density estimation is needed, nonpara-metric methods are often called upon-most notably the popular kernel density estimation (KDE) method. However, the practitioner is instantly faced with the formidable computational cost of KDE for appreciable dataset sizes, which becomes even more prohibitive when many models with diierent kernel scales (bandwidths) must be evaluated { this is necessary for nding the optimal model, among other reasons. In previous work we presented an algorithm for fast KDE which addresses large dataset sizes and large dimensionalities, but assumes only a single bandwidth. In this paper we present a generalization of that algorithm allowing multiple models with diierent bandwidths to be computed simultaneously, in substantially less time than either running the single-bandwidth algorithm for each model independently , or running the standard exhaustive method. We show examples of computing the likelihood curve for 100,000 data and 100 models ranging across 3 orders of magnitude in scale, in minutes or seconds. 1 KERNEL DENSITY ESTIMATION Density estimation. In situations where the density of a dataset itself (rather than other inferences) is of importance, such as in exploratory scientiic data analysis , nonparametric methods are used because they make minimal or no assumptions about the distribution of the data, while achieving high accuracy { making serious density estimation almost synonymous with nonparametric methods 14, 12]. For example, kernel density estimation (KDE), the most widely used and studied nonparametric density estimation method and thus our focus here, can be shown to converge to the true underlying density with probability 1 as more data are observed, with no distribution assumptions at all, requiring only mild conditions on the kernel function and scale 2]. Highly-accurate density estimation is also of use as a core engine in probabilis-tic learning tasks, from classiication to regression to clustering (though arguably for some problems density estimation itself may be skipped and the problem better solved directly, e.g. in discriminative classiica-tion 16]). Unfortunately, the inherent exibility which yields these beneets comes at a very high computational cost, which is our primary focus in this paper. Kernel density estimation. The task is to estimate the density ^ p(x q) for each point x q in a query (test) dataset X Q (having size N Q), from which we can also compute the overall log-likelihood of the dataset b L Q = P NQ q=1 log ^ p(x q). The 'model' is the training …